System for reordering sequenced based packets in a switching network

ABSTRACT

System for reordering sequenced based packets in a switching network. The system includes time stamp logic that operates to associate a receive time indicator with each received data packet. The system also includes Enqueue logic that operates to compute an expiration time for each received packet based on the receive time indicator, and stores the expiration time and the sequence identifier for each received packet into a table. Dequeue logic operates to read the table to determine the received data packets to output in an output stream so that the received data packets are output in a selected order. The Dequeue logic also operates to determine a true expiration time for one or more unreceived data packets, and if the true expiration time for a selected unreceived data packet is reached, the Dequeue logic operates to omit the selected unreceived data packet from the output stream.

FIELD OF THE INVENTION

[0001] The present invention relates generally to the operation ofswitching networks, and more particularly, to a system for reorderingsequence based packets in a switching network.

BACKGROUND OF THE INVENTION

[0002] Communications networks are now required to handle data at veryhigh data rates. For example, a data rate of 10 gigabits per second(Gbps) is common. When it is required to process data at these speeds,multiple high-speed parallel connections may be used to increase theeffective bandwidth. However, this may result in one or moretransmission problems, since the data streams must be divided to bedistributed over the multiple parallel connections, and then at somepoint after parallel transmission, recombined to form the originalstreams.

[0003]FIG. 1 shows a block diagram 100 of a typical network structurefor transmitting data frames (or data packets) from source processors102 to a destination processor 104 via multiple communication fabrics106. The data streams include packets or frames that may comprise afixed amount of data. For example, stream A may include frames A0, A1,and A2 that are received by the source processor A and transmitted toeach of the fabrics as shown. The stream B may include frames B0, B1 andB2 that are transmitted to the fabrics by source processor B as shown,and the stream C may include frames C0, C1 and C2 that are transmittedto the fabrics by source processor C as shown.

[0004] Once the frames are received by the fabrics, they are transmittedto the destination processor 104 as shown. The destination processorreceives the frames and combines them in the order they arrive to formoutput stream D for transmission to another destination. In somesystems, the destination processor breaks up stream D into multiplestreams and transmits the multiple streams to another destination viamultiple communication fabrics.

[0005] A significant problem that exists with current transmissionsystems, such as the system shown in FIG. 1, is that the frames may endup in the wrong order when stream D is formed and transmitted from thedestination processor D. For example, if frames are originallytransmitted via multiple transmission paths, they might arrive at thedestination in an order that is different from how they weretransmitted. For example, the frames may be output in stream D in theorder shown at 108. In this case, frame B2 is output before frame B1,and frame C2 is output before frame C1. Thus, the frames for sourceprocessors B and C are transmitted out of order. In such a case, it maybe necessary to discard out of order frames of data and request a newtransmission of those frames. As a result, additional overhead will beused and a corresponding loss of transmission bandwidth will berealized.

[0006] Another problem that may occur in current transmission systemsinvolves frames that may be lost in transmission, so that they neverarrive at the destination processor. In this situation, it becomesdifficult to determine when a frame is lost or merely delayed intransmission. Furthermore, if a frame is lost, the transmission of thereceived frames may be excessively delayed or blocked because of delaysassociated with the lost frame. Thus, the received frames pay a penaltyfor the lost or delayed frames in that the transmission of the receivedframes may be delayed or blocked completely.

[0007] Therefore, it would be desirable to have a system to reorderframes of data in a transmission system so that the frames are output inthe correct order, thereby improving transmission efficiency. The systemshould also provide a way to process lost or delayed frames so that thetransmission of received frames at a destination processor is notexcessively delayed or blocked.

SUMMARY OF INVENTION

[0008] The present invention includes a system for reordering sequencedbased packets in a switching network. For example, in one embodiment ofthe invention, a reordering system is provided that receives packetsfrom multiple sources and reorders the received packets based on aunique sequence number and arrival time associated with each packet. Inanother embodiment of the invention, the packets are further providedwith a priority, and the priority is used to determine the order thatthe packets are output.

[0009] In one embodiment included in the present invention, a system isprovided for reordering data packets at a destination processor in apacket switching network, wherein a plurality of source processorstransmit the data packets to the destination processor via one or morecommunication fabrics, and wherein a sequence identifier is associatedwith each data packet. The system comprises time stamp logic thatoperates to associate a receive time indicator with each received datapacket. The system also comprises Enqueue logic that operates to computean expiration time for each received packet based on the receive timeindicator, and to store the expiration time into a table based on thesequence identifiers associated with the received packets. The systemalso comprises Dequeue logic that operates to read the table todetermine the received data packets to output in an output stream, sothat the received data packets are output in a selected order, whereinthe Dequeue logic also operates to determine a true expiration time forone or more unreceived data packets, and wherein if the true expirationtime for a selected unreceived data packet is reached, the Dequeue logicoperates to omit the selected unreceived data packet from the outputstream.

[0010] In another embodiment included in the present invention, a methodis provided for reordering data packets at a destination processor in apacket switching network, wherein a plurality of source processorstransmit the data packets to the destination processor via one or morecommunication fabrics, and wherein a sequence identifier is associatedwith each data packet. The method comprises the steps of associating areceive time indicator with each received data packet, computing anexpiration time for each received packet based on the receive timeindicator, storing the expiration time and the sequence identifier foreach received packet into a table, reading the table to determine thereceived data packets to output in an output stream so that the receiveddata packets are output in a selected order, determining a trueexpiration time for one or more unreceived data packets, and omitting aselected unreceived data packet from the output stream if the trueexpiration time for the selected unreceived data packet is reached

BRIEF DESCRIPTION OF DRAWINGS

[0011] The foregoing aspects and the attendant advantages of thisinvention will become more readily apparent by reference to thefollowing detailed description when taken in conjunction with theaccompanying drawings wherein:

[0012]FIG. 1 shows a block diagram of a typical network structure fortransmitting data packets from source processors to destinationprocessors;

[0013]FIG. 2 shows a diagram of one embodiment of a system forreordering packets constructed in accordance with the present invention;

[0014]FIG. 3 shows a diagram of one embodiment of a reordering processorconstructed in accordance with the present invention;

[0015]FIG. 4 shows a diagram that illustrates the computation of a trueexpiration time for a data packet associated with a sequence number inaccordance with the present invention;

[0016]FIG. 5 shows a flow diagram of one embodiment of a method forreordering packets in accordance with the present invention; and

[0017]FIG. 6 illustrates an example of reordering packets in accordancewith the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENT

[0018] The present invention includes a system for reordering sequencedbased packets in a switching network. For example, in one embodiment ofthe invention, a reordering system is provided that receives packetsfrom multiple sources and reorders the received packets based on a timestamp associated with each received packet. Thus, various embodiments ofthe system included in the present invention are discussed in detail inthe following text.

[0019] Exemplary Embodiment

[0020]FIG. 2 shows a diagram of one embodiment of a system 200 forreordering frames constructed in accordance with the present invention.The system 200 includes sequence encoders 202 at each source processorthat encode packets in each stream with a unique sequence identifierbefore transmission via fabrics 210. The system also includes areordering system 204 constructed in accordance with the presentinvention that includes time stamp logic 206 that operates to associatea received time stamp with each received packet. The reordering system204 operates to output packets at output stream D that are in thecorrect order with respect to how they were transmitted from theirrespective source processors. Furthermore, the reordering system usesthe time stamp logic 206 to make a determination about lost or delayedpackets so that these packets do not excessively delay or blocktransmission of received packets from the destination processor D′.

[0021] During operation of the system 200, the source processors receivedata streams containing data packets or frames. The source processorsoperate to give each of the data frames a unique sequence identifierprior to transmitting them to one or more fabrics 210. For example, thesequence identifier may be a sequence number and source identifier thatuniquely identifies the packet and its transmitting source. In oneembodiment of the invention, the source processors also assign apriority to each frame in addition to the sequence identifier. Thus,each frame that is transmitted to the fabrics 210 includes a sequenceidentifier, and optionally, priority information. Any technique can beused to encode a sequence identifier and/or priority with the dataframes. For example, the source processors can include encoder logicthat can encode each data frame with a unique sequence number that actsas the sequence identifier. Furthermore, the encoder logic can include apriority indicator with each frame. The priority indicator can beselected from one of several priority levels. Thus, any suitabletechnique can be used within the scope of the invention to associate asequence identifier and priority information with each data frame.

[0022] The system 200 also includes the reordering system 204 at thedestination processor D′. The reordering system 204 operates to receivethe frames from the fabrics 210 and process the frames based on theirrespective sequence identifiers to reorder the frames for output fromthe destination processor D′. The frames are reordered so that they areplaced in an identical order as when they were transmitted from theirrespective source processors. Thus, stream D′ will include all receivedframes in correct order.

[0023] The reordering system 204 also operates to assign an expirationtime to unreceived frames based on receive time parameters associatedwith received frames. The expiration time is used to determine whenunreceived frames are lost, so that the transmission of received framesas part of reordered stream D′ will not pay a penalty by beingexcessively delayed or blocked.

[0024]FIG. 3 shows a diagram of one embodiment of a reordering system300 constructed in accordance with the present invention. The reorderingsystem forms part of the destination processor D′ and operates toprovide reordering of received frames in accordance with the invention.In addition to reordering in accordance with the present invention, thedestination processor D′ may operate on the data frames in other ways tofacilitate their transmission, however, these other processes are notessential to the operation of one or more embodiments of the invention,and so will not be described in detailed. For example, the destinationprocessor D′ may serialize the frames for transmission, or provide knownerror detection and correction processes that are independent from thereordering system.

[0025] A receiver 302 receives one or more data streams 304 that havearrived at the destination processor D′. For example, the receiver 302receives the frames transmitted via the fabrics 210 from the sourceprocessors. The receiver 302 is coupled to a memory 306 so that each ofthe received frames may be stored in the memory 306.

[0026] Time stamp logic 308 is coupled to the receiver 302 and operatesto provide a time stamp for each received data frame. Information abouteach received frame and its associated time stamp are then stored in atable 310 for later processing.

[0027] Enqueue logic 312 is coupled to the memory 306, so that theEnqueue logic 312 can control the process of loading the memory 306 withthe frames as they are received and also process information in thetable 310. The Enqueue logic may comprise any suitable hardware such asa CPU, gate array or other hardware logic, and may also include anysuitable software to operate in conjunction with the hardware. TheEnqueue logic 312 operates to load expiration times for received packetsinto the table 310 for later use during operation of the reorderingsystem.

[0028] The memory 306 is coupled to a transmitter 314 that receives theframes as they are transferred out of the memory 306 in the correctorder in accordance with the present invention. Thus, as the data framesare received, they are time stamped and their data is entered intomemory, and as a result of the operation of one or more embodiment ofthe present invention, the data will flow from the memory to thetransmitter 314 in the correct order for transmission to anotherdestination. In one embodiment, the transmitter 314 transmits the framesin a single stream 316 toward their final destination. For example, thetransmitter may transmit the single stream 316 into another transmissionfabric. In another embodiment, the transmitter 314 operates to break-upthe reordered stream 316 for transmission to another destination viaseveral transmission fabrics.

[0029] Dequeue logic 316 is coupled to the memory 306 and operates tocontrol the reordering of frames and the transfer of frames from thememory to the transmitter 314. For example, in one embodiment, theDequeue logic operates to control the transfer of frames from the memorybased on the sequence number and time stamp associated with each framethat is stored in the table 310. The Dequeue logic may comprise anysuitable hardware such as a CPU, gate array or other hardware logic, andmay also include any suitable software to operate in conjunction withthe hardware.

[0030] The Dequeue logic also provides a timing function that candetermine when unreceived frames have expired. For example, the Dequeuelogic may wait for a selected time period before determining that anunreceived frame has been lost in transmission. In one embodiment, thistime period provides a way to compensate for a latency time associatedwith the transmission fabrics, thus allowing frames that have been onlyslightly delayed during transmission through the transmission fabrics tobe included in the reordered output stream. In another embodiment, theselected time period allows data frames that are either lost orexcessively delayed to be declared expired, so that the Dequeue logicoperates to output other frames received within acceptable timeparameters. Thus, the reordering system 300 operates to performreordering in accordance with the present invention and to prevent lostframes from excessively delaying the continued transmission of receivedframes.

[0031]FIG. 4 shows a diagram 400 that illustrates the computation of atrue expiration time for unreceived data packets in accordance with thepresent invention. The algorithm for determining the true expirationtime is based on the following two conditions.

[0032] 1. As packets are sent to the destination, they are sequenced inorder.

[0033] 2. If a packet with sequence number S+1 arrives at time T, thenthe packet with sequence number S must have either arrived or willarrive no later than time T+d, where d is the delay through thecommunication fabric.

[0034] As packets arrive at the destination, their data is stored awayinto a memory and a table 401 is used to track their reception based ontheir sequence number. The packets are time stamped upon reception andthe arrival time of received packets is noted and a maximum timeoutperiod is added to that arrival time to determine an expiration time forunreceived packets having earlier sequence numbers. For example, theresultant expiration time associated with a selected received packet isextended down to every packet having a earlier sequence number from thatreceived packet. The process of extending time down to the locationsthat precede the current packet indicate that for every non-valid tableentry (i.e., packet not received), if the time at the destination wereto equal the timeout value, it could be safely assumed that the packetassociated with this entry will never arrive. Since packets may arrivecompletely out of sequence, and since this extension is done on a perpacket basis, a true expiration time for the non-valid entries isdetermined by taking a minimum (MIN) function of all the times thatextend beyond those non-valid entries.

[0035] The following text describes the process of determining a trueexpiration time for non-valid entries in the table 401, (i.e., forpackets that have not been received at the destination). The followingcomputation assumes a timeout period that is equal to ten, however, anydesirable timeout period may be selected. To determine expiration timesassociated with received packets, the time stamp associated with thereceived packets are used. For example, referring to FIG. 4, a validpacket is received at time (4) having sequence number twenty-one, asshown at 402. An expiration time for that packet is determined by addingthe timeout period to the received time to determine an expiration timeequal to fourteen. This expiration time is extended down to packetshaving sequence numbers less than 21 (i.e., 20, 19, 18, etc.) as shownat 404.

[0036] The next packet to arrive has sequence number (28), which arrivesat time five, as shown at 406. As a result, an expiration time offifteen (5+10) is determined and extended down to packets havingsequence numbers less than 28 as shown at 408.

[0037] Another packet arrives at time (6) having sequence numberthirty-two, as shown at 410. This received packet results in anexpiration time of sixteen (6+10) being extended down to packets havingsequence numbers less than 32, as shown at 412. Similarly, a packetreceived at time (8), shown at 414, and having a sequence numberthirty-five establishes an expiration time of eighteen (8+10) forpackets having sequence numbers less than 35, as shown at 416. Anotherpacket arrives at time (12), as shown at 418, and has a sequence numberof twenty-five. This received packet results in an expiration time oftwenty-two (12+10) being assigned to packets having sequence numbersless than 25, as shown at 420.

[0038] To determine a true expiration time for non-valid table entries(i.e., unreceived packets), a MIN function is used so that for eachnon-valid entry, the minimum downward extended expiration time isselected as the true expiration time. For example, the minimum downwardextended expiration time for sequence numbers 34-33 is 18, so the trueexpiration time for those unreceived packets is 18, as shown at 422. Theminimum expiration time for sequence numbers 31-29 is 16, so the trueexpiration time for those unreceived packets is 16, as shown at 424. Ina similar fashion, the minimum expiration time for the remainingunreceived packets can be used to determine their respective trueexpiration times. For packets 27, 26, 24, 23, and 22, the trueexpiration is 15, as shown at 426. For packets 20-18 the true expirationtime is 14, as shown at 428. Therefore, as expiration times are set andextended downward for each received packet, and true expiration timesfor non-valid table entries can be determined using the MIN function.

[0039]FIG. 5 shows a flow diagram 500 for one embodiment of a method forreordering frames in accordance with the present invention. At block502, streams of packets are received at one or more source processorsfor transmission over one or more fabrics to at least one destinationprocessor. At block 504, the packets received at each source processorare given a unique sequence number, and optionally prioritized. Forexample, the sequence number may include a source identifier so thateach packet may be uniquely identified.

[0040] At block 506, the sources transmit their respective streams ofsequenced packets, via one or more fabrics, to at least one destinationprocessor. At block 508, the streams of sequenced packets are receivedat the destination processor. Each received packet is provided with atime stamp that indicates a receive time for the packet.

[0041] At block 510, the sequence numbers, source identifier and anypriority information is extracted from the received packets. At block512, the packet data is stored into memory at the destination processor.A memory locator is associated with each packet, so that the packet datamay be retrieved from the memory.

[0042] At block 514, a memory table is updated to map the arrival of thepackets, their respective sequence numbers, expiration times, and memorypointers that identify the locations in the memory where the dataassociated with each packet is stored.

[0043] The operations discussed with respect to blocks 502 through 514are repeated so that as additional packets are received from the sourceprocessors, their data is stored in memory and the memory table isupdated in accordance with the above steps. The remaining steps in themethod 500, shown below, are used to reorder the received packets inaccordance with the present invention and transmit the received packetsin the correct order to another destination.

[0044] At block 516, a determination is made to determine a packet to beunloaded from the memory for output. The packet is determined from thememory table based on sequence number, and optionally, a priority level.For example, in one embodiment, the packets are determined from theirassociated sequence numbers without regard to priority levels. Inanother embodiment, the packets with higher priority are output insequence before packets with lower priority.

[0045] At block 518, a test is performed to determine if the determinedpacket is available for output. If the packet has not arrived, themethod proceeds to block 520 where a test is performed to determine ifthe true expiration time for the packet has been reached by the currenttime. For example, the destination processor has a time source that isused to keep track of current time and to time stamp received packets.The true expiration time for any unreceived packet includes a selectabletransmission factor that represents a transmission time through thecommunication fabric that the packet flows through to get to thedestination. Thus, if the current time exceeds the true expiration timefor the determined packet, and that packet has not been received, thepacket is considered to be lost.

[0046] If the expiration time has not been reached by the current time,the method proceeds back to block 518 to continue checking to see if thepacket has arrived. If the expiration time has been reached by thecurrent time, then it is assumed that the packet has been lost, and themethod proceeds to block 522 where the packet is dropped and a nextpacket is determined for output at block 516.

[0047] At block 524, assuming the packet is available, the selectedpacket is unloaded from the memory, and the freed memory is madeavailable to store newly received data packets. For example, the memorypointer associated with the packet is reused by the Enqueue logic tostore a newly received packet. At block 526, the selected packetretrieved from the memory is output from the reordering system toanother destination. The method then proceeds to block 516 where anotherpacket is determined for output in accordance with the presentinvention.

[0048] In accordance with the present invention, frames transmitted froma particular source are reordered to have the same order as wheninitially transmitted. Therefore, one or more embodiments included inthe present invention provide a system for reordering frames in aswitching network, so that lost frames can be determined, and as aresult, the transmission of received frames is not excessively delayed.

[0049] Exemplary Operation

[0050]FIG. 6 illustrates an example of one embodiment for reorderingpackets in accordance with the present invention. To demonstrate theoperation of this embodiment of the invention, it will be assumed that astream of packets transmitted from a particular source processor isreceived at a destination processor and reordered in accordance with thepresent invention. Although one or more embodiments included in thepresent invention operate to reorder packets received from multiplesource processors that may have multiple priority levels, for thepurposes of clarity, the present example demonstrates reordering ofpackets received from a single source processor. However, it will beapparent to one with skill in the art that the embodiments of theinvention described herein can be expanded, modified, or adjusted toreorder packets received from multiple source processors.

[0051] In one embodiment, the reordering system may be implemented in asingle integrated circuit (IC chip), where tables and/or memory that areused to reorder the packets are implemented in an off-chip memory (SDRAMor SRAM). However, it is also possible to implement the tables and/ormemory on-chip.

[0052] Referring now to FIG. 6 there is shown Enqueue logic 312, Dequeuelogic 316, time logic 308, and memory 602 with associated address FIFO604. Also shown in FIG. 6 is memory table 310. In the presentembodiment, the memory 602, FIFO 604 and table 310 are part of thememory 306 of FIG. 3. However, in accordance with the invention, theseelements may be formed from the same memory or separate memories, or maybe formed from any other combination of logic, memory or registers.

[0053] There are four registers that are also part of the reorderingsystem shown in FIG. 6. The four registers are associated with eachtransmitting source processor for reordering packets associated witheach respective source processor. For clarity, the registers are shownas individual registers, however, the registers may be part of theEnqueue logic 312, the Dequeue logic 316 or implemented in any otherhardware or software associated with the reordering system. The fourregisters are defined as follows.

[0054] 1. A read pointer (read_ptr) register 606 points at a currentlocation in the table 310 and is used when forming the reordered outputstream.

[0055] 2. A delayed store (delayed_store) register 608 stores parametersassociated with incoming packets until the parameters are ready to bewritten into the table 310.

[0056] 3. A current value (current_value) register 610 holds an ongoingcurrent sequence number and expiration time used when forming thereordered output stream.

[0057] 4. A largest value (largest_value) register 612 holds the largestsequence number and expiration time seen while reading parameters fromthe table 310 when forming the reordered output stream.

[0058] As packets arrive at the destination processor, the Enqueue logic312 obtains a memory pointer from the address FIFO 604 and uses thispointer to store the packet's data into the memory 602. As packets enterthe reorder system, a time-stamp will be associated with each packet aswill a timeout-period (this timeout-period is under software control andcan be modified for any reason). For example, the time out periodrepresents the time it is expected to take for a packet to flow from thesource to the destination via the communication fabric.

[0059] An expiration time is calculated by adding the time-stamp andtimeout-period together. The sequence number of the packet andexpiration time are packet parameters that are referred to as thedata-pair of the packet. The data-pair of the packet are stored in thetable 310 at a table location indicated by a current sequence numberthat is contained in the delayed_store register 608. The table locationpointed to by the delayed_store register represents a pointer from theprevious packet.

[0060] To illustrate the above process, it will be assumed that a streamof packets is received at the reordering system, and sequence numbers,time stamps and expiration times for a portion of those packets areshown at 614. It will be assumed that the packet with sequence number(5) has been processed by the Enqueue logic of the reordering system.When this packet is processed, its sequence number and memory pointerare stored into the delay_store register 608. The next packet to beprocessed has sequence number (18) as shown at 616.

[0061] The Enqueue logic obtains a memory pointer from the address FIFO604 and uses this pointer to store the data associated with packet (18)into the memory 602. The Enqueue logic then stores the data-pair(seq/exp) for packet 18 in the table 310 at the address pointed to bythe delay_store register. Thus, the data-pair for packet 18 is stored atlocation (5) in the table 310 as shown at 618. Also stored at location(5) in the table 310 is the memory pointer for packet (5) data. TheEnqueue logic then stores the sequence number and memory pointer forpacket (18) into the delay_store register 608 as shown at 620. Thus, thenext packet processed by the Enqueue logic will have its parametersstored at location (18) in the table 310.

[0062] To summarize the above process, the data pointer for the packethaving sequence number (5) that is stored in the table 310 is correctfor that sequence number, but the accompanying data-pair points to thenext packet that has arrived. At the same time, the new packet (18) hasits sequence number and memory pointer stored into the delayed_storeregister 608. Thus, the delayed_store register lives-up to its name bydelaying stores into the table 310 until a subsequent packet isreceived. The delayed store register also skews where the stores happen(i.e. addressing) so as to adhere to the underlying algorithm includedin the present invention.

[0063] The above process is used by the reordering system included inthe present invention to receive packets, time stamp the packets,compute an expiration time for the packets, store the packet data intomemory, and store packet parameters into the table 310. The followingtext will describe how the reordering system operates to retrieve thepacket data from memory and reorder the packets to form a reorderedoutput stream in accordance with the present invention.

[0064] The Dequeue logic operates to retrieve the packet data from thememory 602 using the read-ptr register 606 and the table 310 to form areordered output stream 632. The read_ptr register points to the nextsequence number in the table 310 to determine the packet data to beretrieved from the memory. If the value read from the table at theselected sequence number is valid, its accompanying memory data pointeris used to retrieve the data from memory for output. If the value fromthe table is not valid, the Dequeue logic operates to determine if thepacket associated with that sequence number has expired.

[0065] To determine whether the packet associated a sequence number hasexpired, the Dequeue logic compares the current time (system time) fromthe time logic 308 to the expiration time stored in the current_valueregister 610. If the expiration time stored in the current_valueregister is greater than the system time, the Dequeue logic operates todetermine if the packet has been received and is being processed by theEnqueue logic. Thus, the packet has not expired and the system operatesto wait to receive that packet. However, once the system timer equalsthe expiration time and the packet has still not arrived, the Dequeuelogic deems that packet to be lost, and the read_ptr is incremented topoint to the next entry in the table to find the next sequenced packetto output. If the next entry is not valid, a comparison of expirationtime (which hasn't changed because no valid packet has been read) andthe system time will indicate that that entry is lost as well. Thisprocess continues until a valid entry is read.

[0066] The current_value register holds the next data-pair (i.e., thenext sequence number and expiration time) that can be used to define thetrue expiration time for one or more packets. For example, entries forthe current_value register for the above example are shown at 622. Thecurrent_value register is updated when the read_ptr equals thecurrent_value sequence number. For example, when the read_ptr points tosequence number (18) in the table 310, the current value register isupdated to have a sequence number of (20) and an expiration time of(19), as shown at 624.

[0067] If the next sequence value read from the table is less than thecurrent_value's sequence number, the largest_value register 612 is usedto determine the entries in the current_value register 610. Thisprevents a situation where a previously accessed sequence number'sexpiration time is used. For example, when the read_ptr is at sequencenumber (20) the sequence number read from the table is (10). This numberis less than the value in the current_value register. In this case, thenext entries for the current_value register are determined from thelargest_value register 612. Thus, the sequence number (40) andexpiration time (25) are entered into the current_value register, asshow at 626.

[0068] The largest_value register holds the largest sequence number readthus far from the table. As each entry in the table is read, thisregister is updated with every sequence number that is higher than thecurrent contents of the register. The associated expiration time is alsoentered. Therefore, the use of the current_value register and thelargest_value register operate to provide true expiration times for thepackets, as shown at 628. Upon reset, the current value register willequal the first read_out value pair.

[0069] There are two valid bits (v1 and v2), shown at 630, that are partof the table 310 and are used to reduce the number of writes that arenecessary to invalidate entries as they are read. A selected column, forinstance v1, is used to indicate whether or not the current table entryis valid, and the other column, v2 in this case, is reset when v1 iswritten. For example, referring to table 310, v2 is used to indicatethat current table entries are valid when a “1” is present. At the sametime, v1 is set to “0.” When enough frames are received so that thetable fills and eventually wraps-around, v1 will then be set to “1” whena new entry is valid, and v2 will be reset to “0.” Thus, by alternatingbetween v1 and v2 to indicate when a table entry is valid (i.e., apacket has arrived), it is only necessary to perform one write operationto validate an entry, which eliminates the need to perform a secondwrite operation to invalidate the entry when the corresponding packet isoutput.

[0070] There is a global setting that indicates which valid bit (v1 orv2) to use to indicate that a table entry is valid. For example, theglobal setting is controlled by the Enqueue logic 312. In this way, whenall the table entries are exhausted, the system changes the globalsetting to reverse the role of the valid bits (v1, v2) so that theDequeue logic knows to look at the correct valid bit to determine when atable entry is valid. Therefore, an extra write cycle is not needed toinvalid an entry.

[0071] As the Dequeue logic 316 reads the table 310 to obtain addresspointers for the packet data to be output, those address pointers arereturned to the FIFO 604 so that they may be reused by the Enqueue logic312 to store newly received packet data. By using the read_ptr registerto sequentially read the table 310 to obtain address pointers toreceived packet data, the Dequeue logic operates to output the packetdata in the correct order with respect to how it was transmitted fromthe originating source processor. The Dequeue logic also operates todetermine true expiration times for unreceived packets and skips thosepackets when it is determined that they have expired, thereby preventinglost packets from unreasonably delaying or blocking the formation of theoutput stream 632.

[0072] Pseudo Code

[0073] To further illustrate the inter-workings of the reorderingalgorithm included in the present invention, the following pseudo-codeis presented that describes how the current_value and largest_valueregisters are updated in accordance with the present invention.

[0074] Current_Value Register

[0075] To initialize this register, after a reset occurs the sequencenumber included with the first incoming packet is entered as thecurrent_value. The following code is used to update the current_valueregister after initialization.

[0076] if (read_ptr==current_value.sequence_number)

[0077] current_value.sequence_number=read_out.sequence_number

[0078] current_value.expiration_time=read_out.expiration_time

[0079] if (read_ptr=current_value.sequence_number)

[0080] if (current_value.sequence_number>read_out.sequence_number)

[0081] current_value.sequence_number=largest_value.sequence_number

[0082] current_value.expiration_time=largest_value.expiration_time

[0083] if ((read_ptr==current_value.sequence_number) & !valid)

[0084] send delay_store register

[0085] Largest_Value Register

[0086] The largest_value register operates as a place holder for thehighest sequence numbers read from the table during operation of thereordering system. The following code is used to update thelargest_value register after initialization.

[0087] if (read_out.sequence_number>largest_value.sequence_number)

[0088] largest_value.sequence_number=read_out.sequence_number

[0089] largest_value.expiration_time=read_out.expiration_time

[0090] Although described with reference to the utilization of the table310, it will be apparent to one with skill in the art that a separatetable, memory, or memory map may be used to process data packets fromeach transmitting source processor, and if required, for each respectivepriority used. Thus, in other embodiments of the present invention, theEnqueue logic and the Dequeue logic are coupled to multiple tables, andprocessing steps similar to those described above are performed for eachtable to reorder the received data packets for output in the reorderedstream 632.

[0091] The present invention includes a system for reordering sequencedbased packets in a switching network. The embodiments described aboveare illustrative of the present invention and are not intended to limitthe scope of the invention to the particular embodiments described.Accordingly, while several embodiments of the invention have beenillustrated and described, it will be appreciated that various changescan be made therein without departing from the spirit or essentialcharacteristics thereof. Accordingly, the disclosures and descriptionsherein are intended to be illustrative, but not limiting, of the scopeof the invention, which is set forth in the following claims.

What is claimed is:
 1. A system for reordering data packets at adestination processor in a packet switching network, wherein a pluralityof source processors transmit the data packets to the destinationprocessor via one or more communication fabrics, and wherein a sequenceidentifier is associated with each data packet, the system comprising:time stamp logic that operates to associate a receive time indicatorwith each received data packet; Enqueue logic that operates to computean expiration time for each received packet based on the receive timeindicator, and to store the expiration time into a table based on thesequence identifiers associated with the received packets; and Dequeuelogic that operates to read the table to determine the received datapackets to output in an output stream, so that the received data packetsare output in a selected order, wherein the Dequeue logic also operatesto determine a true expiration time for one or more unreceived datapackets from the expiration times associated with received packets, andwherein if the true expiration time for a selected unreceived datapacket is reached, the Dequeue logic operates to omit the selectedunreceived data packet from the output stream.
 2. The system of claim 1,further comprising: a memory coupled to the Enqueue logic; and a pointerFIFO coupled to the Enqueue logic, wherein the Enqueue logic operates toobtain a pointer to the memory from the pointer FIFO for each receiveddata packet and stores the received data packets in the memory based onthe pointers.
 3. The system of claim 2, wherein the memory and thepointer FIFO are coupled to the Dequeue logic.
 4. The system of claim 3,further comprising a read pointer register that is used to read selectedentries from the table.
 5. The system of claim 4, further comprising adelay store register that is used to store the pointers into the table.6. The system of claim 5, further comprising a current value registerthat is used to determine the true expiration time.
 7. The system ofclaim 6, further comprising a largest value register that is used tomaintain a largest sequence identifier value.
 8. A method for reorderingdata packets at a destination processor in a packet switching network,wherein a plurality of source processors transmit the data packets tothe destination processor via one or more communication fabrics, andwherein a sequence identifier is associated with each data packet, themethod comprising steps of: associating a receive time indicator witheach received data packet; determining an expiration time for each ofthe received data packets based on their associated receive timeindicators; determining a true expiration time for one or moreunreceived data packets based on the expiration times associated withthe received data packets; forming an output stream of received datapackets, wherein the output stream contains the received data packets inthe same order that they were transmitted from their respective sourceprocessors; and omitting unreceived data packets from the output streamif the true expiration time for the unreceived data packets is reached.9. The method of claim 8, further comprising steps of: storing theexpiration time and the sequence identifier for each received packetinto a table; and reading the table to determine the received datapackets to output in the output stream.
 10. The method of claim 9,further comprising steps of: obtaining a memory pointer from an addressFIFO for each received packet; and storing each received packet in amemory based on the associated memory pointer.
 11. The method of claim10, further comprising a step of returning the memory pointer associatedwith each packet in the output stream to the address FIFO.